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A SYSTEM AND METHOD FOR STATISTICALLY COMPARING 
AND MATCHING PLURAL SETS OF DIGITAL DATA 

RArKPiROi INn OF THF INVFNTinN 

1 . Field of the Invention. 

The present invention relates in general to object detection and 
tracking, and in particular to a system and method for statistically 
comparing and matching plural sets of digital data. 

2. Related Art. 

Applications for automatic digital object detection and tracking, 
image registration, pattern recognition and computer vision analysis are 
becoming increasingly important for providing new classes of services to 
users based on assessments of the object's presence, position, trajectory, 
etc. These assessments allow advanced and accurate digital analysis 
(such as pattern recognition, motion analysis, etc.) of the objects in a 
scene, for example, objects in a sequence of images of a video scene. 
Plural objects define each image and are typically nebulous collections of 
pixels, which satisfy some property. These pixels could be the result of 
some pre-processing operation such as filtering, equalization, edge or 
feature detection, applied to raw input images. Each object can occupy a 
region or regions within each image and can change their relative locations 
throughout subsequent images of the video scene. These objects are 
considered moving objects, which form motion within a video scene and 
can be automatically detected and tracked with various techniques, one 
being template matching. 

Template matching is a class of computer algorithms that is used in 
many digital computer applications, such as image registration, pattern 



recognition and computer vision applications. A template matching 
algoritlim defines a function (for example, a metric) that estimates the 
similarity between sets of digital data. In this case, one set of digital data is 
commonly referred to as a template and another set of digital data is 
5 referred to as an image, wherein the template is typically smaller than the 
image (for instance, the template can be a small portion of the image). In 
computer vision applications, the template usually represents an object of 
the image that is being tracked and detected (located) within the image. 
The object can be located by computing the metric at various locations (u, 
10 v) in the image and determining where the metric is maximized. 

However, many systems that use template matching are not robust 
or flexible enough for advanced image registration, pattern recognition and 
f 1 computer vision applications due to unfavorable tradeoffs of functionality 

IJ for performance (for example, restricting themselves to translations of the 

15 template). Therefore, what is needed is a system and method for comparing 
1^ and matching multiple sets of data by transforming one set of data and 

fll performing statistical analyses on the multiples sets of data. Whatever the 

ill merits of the above mentioned systems and methods, they do not achieve 

I; the benefits of the present invention. 

20 

.91 JMMARY OF THF INVFNTinN 
To overcome the limitations in the prior art described above, and to 
overcome other limitations that will become apparent upon reading and 
understanding the present specification, the present invention is embodied 
25 in a system and method for statistically comparing a first set of digital data 
to at least a second set of digital data and matching the first set of digital 
data to appropriately corresponding portions of the second set of digital 
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data. The first or the second set of digital data can be transformed during 
statistical analysis to enhance statistical analysis of the digital data. 

In one embodiment of the present invention, the system includes a 
host processor executing software that implements an address generator, 

5 an acceptance tester and a statistical comparison processor. The host 

processor controls the entire process and initially renders or rasterizes the 
sets of data. The address generator generates addresses, which can 
reflect a transformation, for the first set of data and the second set of data 
to be compared. The addresses are used by filtering functions to generate 

10 per-pixel values, such as color values. The acceptance tester receives the 
per-pixel values and determines the pixels that are to be used to contribute 
to statistical analysis. The statistical comparison processor statistically 
analyzes the pixels between the first data set and the second data set for 
comparison purposes. The host processor then examines the statistical 

15 comparisons computed by the statistical comparison processor and makes 
further processing decisions. The process repeats until a desired result is 
computed, such as a match or non-match between the data sets. 

Alternatively, the system can be implemented in a three-dimensional 
(3D) graphics rasterizer. In this embodiment, the system includes a frame 

20 buffer (a block of graphics memory that represents the display screen) and 
texture memory (a block of graphics memory that can contain portions of 
the display screen), in addition to the components discussed above. The 
first set of digital data can be stored in the frame buffer while the second 
set of data can be stored in the texture memory. Also, statistical 

25 generation can be performed by the rasterizer, with or without actually 

rendering or writing a 3D digital scene comprised of the digital data to the 
frame buffer. In this embodiment, rasterization and rendering techniques 
and advanced statistical generation and comparison of the present 



invention can be integrated to form a novel video graphics device or 
liardware video card for computer systems. 

The present invention as well as a more complete understanding 
thereof will be made apparent from a study of the following detailed 
description of the invention in connection with the accompanying drawings 
and appended claims. 

BRIEF DESCRIPTION OF THF DRAWINGS 
Referring now to the drawings in which like reference numbers 

represent corresponding parts throughout: 

FIG. 1 is a block diagram illustrating an apparatus for carrying out the 

invention. 

FIG. 2 is an overview flow diagram of the present invention. 
FIG. 3 is a flow diagram of the operation of the present invention. 
FIG. 4 is a general block diagram of the present invention. 
FIG. 5 is a block diagram illustrating one embodiment of the present 
invention. 

FIGS. 6A-6C are graphical images illustrating a working example of 
the present invention. 

DE TAI LED DES C R IPT ION OF T HE INVE NTIO N 
In the following description of the invention, reference is made to the 
accompanying drawings, which form a part hereof, and in which is shown by way 
of illustration a specific example in which the invention may be practiced. It is to be 
understood that other embodiments may be utilized and structural changes may 
be made without departing from the scope of the present invention. 



L Exemplary Operating Fnvironment 

FIG. 1 and the following discussion are intended to provide a brief, 
general description of a suitable computing environment in which the 
invention may be implemented. Although not required, the invention will be 
5 described in the general context of computer-executable instructions, such 
as program modules, being executed by a personal computer. Generally, 
program modules include routines, programs, objects, components, data 
structures, etc. that perform particular tasks or implement particular abstract 
data types. Moreover, those skilled in the art will appreciate that the 

10 invention may be practiced with other computer system configurations, 

including hand-held devices, multiprocessor systems, microprocessor-based 
or programmable consumer electronics, network PCs, minicomputers, 
mainframe computers, and the like. The invention may also be practiced in 
distributed computing environments where tasks are performed by remote 

15 processing devices that are linked through a communications network. In a 
distributed computing environment, program modules may be located on 
both local and remote memory storage devices. 

With reference to FIG. 1 , an exemplary system for implementing the 
invention includes a general-purpose computing device in the form of a 

20 conventional personal computer 100, including a processing unit 102, a 
system memory 104, and a system bus 106 that couples various system 
components including the system memory 104 to the processing unit 102. 
The system bus 106 may be any of several types of bus structures including 
a memory bus or memory controller, a peripheral bus, and a local bus using 

25 any of a variety of bus architectures. The system memory includes read only 
memory (ROM) 110 and random access memory (RAM) 1 12. A basic 
input/output system 1 14 (BIOS), containing the basic routines that help to 
transfer information between elements within the personal computer 100, 
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such as during start-up, is stored in ROM 110. The personal computer 100 
further includes a hard disk drive 1 16 for reading from and writing to a hard 
disk, not shown, a magnetic disk drive 1 1 8 for reading from or writing to a 
removable magnetic disk 120, and an optical disk drive 122 for reading from 

5 or writing to a removable optical disk 124 such as a CD ROM or other optical 
media. The hard disk drive 1 1 6, magnetic disk drive 128, and optical disk 
drive 122 are connected to the system bus 106 by a hard disk drive interface 
126, a magnetic disk drive interface 128, and an optical drive interface 130, 
respectively. The drives and their associated computer-readable media 

10 provide nonvolatile storage of computer readable instructions, data 

structures, program modules and other data for the personal computer 100. 
Although the exemplary environment described herein employs a hard disk, 
a removable magnetic disk 120 and a removable optical disk 124, it should 
be appreciated by those skilled in the art that other types of computer 

15 readable media which can store data that is accessible by a computer, such 
as magnetic cassettes, flash memory cards, digital video disks, Bernoulli 
cartridges, random access memories (RAMs), read only memories (ROM), 
and the like, may also be used in the exemplary operating environment. 
A number of program modules may be stored on the hard disk, 

20 magnetic disk 1 20, optical disk 1 24, ROM 1 1 0 or RAM 112, including an 

operating system 132, one or more application programs 134, other program 
modules 136, and program data 138. A user may enter commands and 
information into the personal computer 100 through input devices such as a 
keyboard 140 and pointing device 142. Other input devices (not shown) may 

25 include a microphone, joystick, game pad, satellite dish, scanner, or the like. 
These and other input devices are often connected to the processing unit 
102 through a serial port interface 144 that is coupled to the system bus 106, 
but may be connected by other interfaces, such as a parallel port, game port 



or a universal serial bus (USB). A monitor 146 or other type of display 
device is also connected to the system bus 106 via an interface, such as a 
video adapter 148. In addition to the monitor 146, personal computers 
typically include other peripheral output devices (not shown), such as 

5 speakers and printers. 

The personal computer 100 may operate in a networked environment 
using logical connections to one or more remote computers, such as a 
remote computer 150. The remote computer 150 may be another personal 
computer, a server, a router, a network PC, a peer device or other common 

10 network node, and typically includes many or all of the elements described 
above relative to the personal computer 100, although only a memory 
storage device 152 has been illustrated in FIG. 1. The logical connections 
depicted in FIG. 1 include a local area network (LAN) 154 and a wide area 
network (WAN) 156. Such networking environments are commonplace in 

15 offices, enterprise-wide computer networks, intranets and Internet. 

When used in a LAN networking environment, the personal computer 
100 is connected to the local network 154 through a network interface or 
adapter 158. When used in a WAN networking environment, the personal 
computer 100 typically includes a modem 160 or other means for 

20 establishing communications over the wide area network 1 56, such as the 

Internet. The modem 160, which may be internal or external, is connected to 
the system bus 106 via the serial port interface 144. In a networked 
environment, program modules depicted relative to the personal computer 
100, or portions thereof, may be stored in the remote memory storage 

25 device. It will be appreciated that the network connections shown are 

exemplary and other means of establishing a communications link between 
the computers may be used. 
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IL Gfinfiral Ovftrvifiw 

In general, the present invention is embodied in a system and method for 
statistically analyzing and comparing a first group of pixels of a defined 
portion of a digital scene, such as an object or template within the digital 
5 scene, to a second group of pixels, such as the entire digital scene or the 
image. The template is matched to appropriately corresponding portions of 
the image that represent the template. During statistical comparison and 
matching of the template and the image, either the first or the second group 
of pixels can be raster transformed. For instance, either the template or 
10 the image can be incrementally rotated or skewed to enhance the statistical 
analyses. 

In particular, first, the system receives digital input, such as the 
images or the templates. This data can then be pre-processed, using a 
variety of techniques such as filtering, equalization, and edge or feature 

15 finding. The system then computes statistics on either transformed images 
or transformed templates. Next, the resulting computed statistics are 
analyzed and new requests based on the resulting statistics are generated 
with different transformations and/or different images until a desired result 
is computed, namely a match or non-match between the template and the 

20 image. 

The present invention can be used as an object detection and tracking 
system for computer vision, motion analysis and pattern recognition 
applications, as a video acceleration system for computer graphics video 
hardware cards, as a video CODEC (coder/decoder), or any other suitable 
25 device that can utilize statistical comparison and matching between sets of 
digital data, such as images. In addition, the present invention can be 
implemented as computer software running on a computer system, as 
instruction sets operating within a microprocessor for a hardware device, 
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such as a computer graphics video card, as computer firmware operating 
within a digital device, such as a digital camera, or any other suitable digital 
processing system. 

FIG. 2 is an overview flow diagram of the system and method of the 
5 present invention. In general, the present invention matches a first set of 
digital data to a second set of digital data by statistically comparing the sets 
of data. Namely, first the system receives a first set of digital data as a first 
input, which can be a template, and/or receives a second set of digital data 
as a second input, which can be an image (step 210). Next, the system 

10 200 raster transforms either the first or the second set of digital data and 
computes statistics on the transformation (step 212). The system then 
analyzes the resulting statistics and makes calculated determinations 
based on the resulting statistics for generating new and different 
transformations on the data (step 214) until a desired result is achieved, 

15 such as a match or non-match between the first and the second set of 
digital data (step 216) or between other sets of data. 

IIL Details of Cnmpnnent .s and Opfiratinn- 

FIG. 3 is a flow diagram of the operation of the present invention and 

20 FIG. 4 is a general block diagram of the present invention. Referring to FIG. 
3 along with FIGS. 4 and 2, first, new sets of data, such as an image and/or 
a template is acquired (step 310) by the system 400 and initialized by the 
host processor 408. The host processor 408 can store the new sets of data 
in the memory devices. For instance, the first set of data, such as the 

25 template can be loaded into a first memory device 410 and the second set of 
data, such as the image can be loaded into a second memory device 412. 
Second, models, such as two-dimensional (2D) or three-dimensional (3D) 
models, are rendered and statistics are accumulated (step 312) by the host 
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processor 410 for the template and the image. Rendering and statistical 
accumulation can be accomplished with an address generator 414 and an 
acceptance tester 41 6, which will be discussed in detail below. 

Rendering involves drawing geometric shapes or primitives, such as 
5 polygons, into dedicated memory. It should be noted that the present 

invention preferably uses triangles as the drawing primitive, although there 
are other primitive types that could be used. In general, a single triangle 
can be rendered by taking three vertices v'o. vi , with the following fields 
sx sy (the screen space coordinates of the triangle in the first memory 
10 device) iu tv rhw (the 2D coordinates of each vertex in the texture, and a 
li perspective term). The address generator 414 interpolates these 

m parameters across the triangle; for each pixel in the first memory device 

y subtended by the triangle in screen space (sx sy), the second memory 

|j device is used to compute an interpolated texture value at the 

111 15 corresponding interpolated texture location. The comparison statistics 
li between the two colors are then gathered, depending on the results of the 

L'L acceptance test. This operation can be considered as a resampling 

IJ operation being applied to the texture, which involves filtering the texture at 

Gl different locations and at different densities. 

20 Third, the host processor 408 reads back resulting statistics (step 

314) from the statistics/comparison device 418 and adjusts the 2D/3D 
models based on the resulting statistics (step 316). Steps 312-316 are 
repeated as long as the desired iterations or quality for matching are not 
achieved (step 318). When the desired iterations or quality for matching 
25 are achieved and if additional images or templates need to be analyzed, 
steps 310-316 are performed (step 320). However, if no additional images 
or templates need to be analyzed after the desired iterations or quality for 
matching are achieved, then the matching or non-matching results are 
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completed (step 322). 

In general, the address generator 414 generates addresses for the 
template and the image that are to be compared. The addresses can reflect 
a transformation, such as combinations of rotations, scales and perspective 

5 transforms, of the template or image. The addresses serve as input to 

filtering functions that read from the images to be compared and generate 
color values (RGBA) and, if present, Z buffer and other per-pixel values. 
These values can be used by the acceptance tester 416 to decide whether 
to allow the pixel to contribute to the statistics. If the pixel is permitted to 

10 contribute, the color values are sent to a statistics/comparison device 41 8 
for statistical analyses and comparison processing. 

The statistics/comparison device 418 can contain variables that are 
updated for each pixel based on the input color values. For instance, in 
one embodiment of the present invention, statistical analyses for 

15 comparing and matching the sets of digital data can be accomplished by 
initially defining a function or metric that estimates the similarity between 
the sets of digital data. In this case, one set of digital data is the template 
and the other set of digital data is the image. The template can represent 
an object of the image that is being tracked and detected (located) within 

20 the image. The object can be located in the image by computing the metric 
at various locations (u, v) in the image and determining where the metric is 
maximized. 

In the following examples, T represents the template image and I 
represents the input image. One metric that can be used is a cross- 
25 correlation coefficient, which measures the similarity between the image 
and the template on an absolute scale in the range [-1 , 1], namely: 
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covariance{I, T) Ex Hy (^(^' ^) " X^(^ " 3^ " v) - //; ) 



where and 07 designate the mean and standard deviation of the image 
and //rand orr designate the mean and standard deviation of the template. 
Example variables and computations that can be tracked by the 
statistics/comparison device 418 are illustrated below. To compute the 
statistic, the following sums can be calculated between the template (T) 
and corresponding pixels in the image (I): 

1) si and zT, the sums of the respective pixel values 

2) sIT, the sum of the product of the pixel values 

3) sl^ and sT^ the sums of the squares of the respective pixel values 

4) Pixel Count, the number of pixels that have been accumulated 

It should be noted that computing these sums can dominate the runtime of 
the pattern recognition or other high-level algorithm that is using the metric. 

Also, a summing metric for template matching can be used that 
involves summing some function of the difference between the image and 
template pixels, for example: 

Xl-Tj,\l-T\<d 
\l -T\, otherwise 



f{I,T) = 



where jis some value less than 20 (for 8-bit unsigned integer pixel data). 
A more flexible variation involves computing sLUT(f{l,T)), the sum of a 
lookup based on a function of the pixel values. Two examples of f(l,T) are 
f(l,T)=l-T or f(l,T)=|l-T| (the difference and absolute difference of the pixel 
values, respectively). 
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As mentioned above, a transform can be applied to either the input 
image or the template, in order to find transformed versions of the template 
object. Typical transformations can be combinations of rotations, scales 
and perspective transforms that are relatively close to identity (to minimize 
the size of the search space). All of the above-described variations share 
the characteristic that pixels from the template are iterated over pixels in 
the image, and a calculation is performed between corresponding pixels. 
The template is typically small compared to the image and static over a 
large number of template matching search probes. 

Several higher-level search strategies can be used to find the best 
transformation parameters for a given template. One involves examining 
all possible combinations of parameters, e.g., examine the template at all 
possible positions, orientations, and scales. Some savings can be 
obtained by working in a multi-resolution hierarchy, i.e., to match a reduced 
size template to a reduced size image, and to then try local variations 
(perturbations) to find a better match. 

Another strategy that can be used involves taking derivatives of the 
matching cost with respect to the unknown parameters, and then using a 
generalized gradient descent technique. In this case, in addition to 
summing intensities, (threshold) intensity differences, and intensity 
products or squares (as in regular, normalized, or robust enhanced cross- 
correlation), also products of horizontal and vertical derivatives are 
accumulated with themselves and with the per-pixel difference. If 
transformations other than translation are being estimated, the number of 
required derivatives and products rise quickly. However, it is possible to 
amortize the computation of derivatives that are more complicated and 
their products by dividing the template up into smaller regions or patches, 
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and only accumulating simpler derivatives and products on a per-pixel 
basis. 

An additional strategy that can be used is to let the unknown 
transformation parameters (or at least their updates) be controlled by the 
motion of vertices embedded in the template. For example, the template 
can be a wireframe mesh of an object that is being tracked. The control 
mesh can be discretized into finer polygons, such as triangles, and the 
triangles surrounding each control vertex can be used to estimate that 
vertex's motion. For reasonably textured templates, convergence will occur 
to the same estimate as the full gradient descent techniques, which 
explicitly model the interaction between vertices controlling a common 
patch or triangle. 

In addition, the alpha values in the input colors can be used to weight 
the statistics. For example, If ai is the template alpha and ai the image 
alpha, then a new a can be derived from these values by selecting one, or 
by doing a weighted blend between them. The resulting a could then be 
used to weight the pixel's contribution to the above-described statistics. 
Among other things, this would allow pixels to be importance-weighted by 
the application. 

V. Working Example : 

The following working example is for illustrative purposes only. FIG. 
5 is a block diagram illustrating one embodiment of the present invention. 
In general, similar to FIG.4, the example system 500 of FIG. 5 includes a 
host processor 508, a first memory device 512, such as a frame buffer, a 
second memory device 510, such as a texture memory device, and a 
graphics processor 513 that includes an address generator 514, an 
acceptance tester 516 and a statistics/comparison processor 518. The 
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frame buffer 512 can be a block of graphics memory that represents a 
display for a computer screen and texture memory 510 can be a block of 
graphics memory that can contain portions of the display screen. In 
addition, the example system 500 of FIG. 5 also includes a statistics enable 

5 switch 520 and an alpha blending device 522. In this example, the graphics 
processor 51 3 resamples either the first or the second set of data to be 
matched to each other using a perspective transformation. 

Preferably, this example system can be used as a three-dimensional 
(3D) graphics rasterizer that can be integrated to form a novel video 

10 graphics device or hardware video card for computer systems. In this 
embodiment, the first set of digital data can be stored in the texture 
memory while the second set of data can be stored in the frame buffer. 
Also, statistical generation can be performed with or without actually 
rendering or writing a 3D digital scene to the frame buffer with the statistics 

15 enable switch 520. For implementation of the computer video graphics 

hardware device, additional core logic can be used to compute the statistics 
and to forward the results back to the host processor 508 upon request. 
FIG. 5 shows computations of the statistics between the texture map and the 
frame buffer image for tracking statistics on the two input pixel values instead 

20 of blending between them and writing the result to the frame buffer. 

Namely, when the statistics enable switch 520 is enabled, the 
graphics processor 513 renders the rasterized information (step 220 of 
FIG. 2) without writing the results to the frame buffer. In contrast, when the 
statistics enable switch 520 is disabled, the graphics processor 513 

25 actually renders or writes the rasterized information to the frame buffer and 
display screen. The alpha blending device 522 allows use of an additional 
(such as a fourth) color component that is not displayed, but that 
corresponds to the opacity of a surface. This provides control of the 



amount of color of a pixel in the source surface to be blended with a pixel in 

the destination surface. 

In one specific embodiment of the example of FIG. 5, the template is 

treated as a texture and the frame buffer an image and the display primitive 
5 for rendering purposes is a triangular polygon. In addition, instead of 

rasterizing the texture into the frame buffer, certain statistics can be recorded 

for normalized correlation or other statistics can be recorded for variations. 

In one example, if the texture is considered a template and the frame buffer 

an image, the graphics processor 513 can be used to resample the 
10 template using a perspective transformation. Also, the graphics processor 

513 can be used to record statistics (sT, si, nil, sT^, for normalized 

correlation, or other statistics for a variation) for later forwarding to the host 

processor. 

The example system 500 is extremely flexible because it has the 
15 ability to intersperse rendering/accumulation commands with accumulator 
readback commands. For example, if a deformable triangular patch is 
being tracked, the triangle can be rendered using smaller triangles (such as 
16), and the accumulator can read back after each of the small triangles 
has been rendered. This allows host processor 508 to compute necessary 
20 derivatives and sums to compute a full affine motion for the triangle. 

Similarly, if a full set of data is being tracked, each triangle's accumulated 
values can be read back in order to compute an independent motion of 
each control vertex. The number of readbacks per rendered model are few 
enough that they should not impose a large burden on a hardware graphics 
25 port of a computer system, such as the exemplary computer system 
depicted in FIG. 1. 

Either the first or the second set of data to be compared with each 
other can be rendered at a number of offsets. This allows the host 
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processor 508 to either explicitly find the best position for the first set of 
data, such as the template, or accumulate the required information to 
analytically compute the best update. The offsets are preferably simple 
integer or fractional (such as V2 pixel) perturbations to the vertices. As 
5 such, it is preferable that the system 500 supports differencing of the 
deformed data (such as the template) and the target data (such as the 
image) at a number of pixel or sub-pixel shifts. For example, the host 
processor 508 can specify the shift amount (for instance c/=1 or c/=]4 pixel), 
to enable accumulation differences not only with the target data, but also 
10 with versions shifted by ±d pixels horizontally and vertically (accumulating 9 
times the statistics). For software implementations, the speed/memory- 
hardware tradeoff is good, where the cost of rasterizing a single pixel is still 
several cycles. 

Another advantage of integrating the rasterization and matching 
15 stages is that the hardware can perform the visibility computation for 3D 
model-based tracking. The 3D model would be rendered once in order to 
compute the z-buffer, and then it would be rendered again to compute the 
(per-triangle) statistics. Rendered pixels which fail the z-buffer test would 
be discarded from the computation, since they are visible. 
20 FIGS. 6A-6C are graphical images illustrating a working example of 

the present invention. FIG. 6A shows an image 600 from which a feature of 
interest 610 is to be extracted. FIG. 6B shows the feature of interest 610 as 
a template 615 extracted from FIG. 6A. The feature 610 is the portion of the 
image 600 that is to be tracked or located in subsequent input images. FIG. 
25 6C shows a subsequent input image 630 that contains a rotated and scaled 
version of the feature 610. The system of the present invention detects the 
feature 610 by transforming either the template 615 or the image 630 (in 
this case, the template 615 is transformed) and gathering statistics 



between the transformed template 615 and the input image 630. A suitable 
rasterizer in accordance with the present invention, as described above, is 
used to transform the template 615. 

Display primitives, in this case triangles, are used to transform the 

5 template 615 and locate it in the input image 630. For instance, the 
mapping of the triangles from template 615 to image 630 is shown by 
arrows 640, 645. Also, although two display primitives encompass the 
entire template 615 for rasterizing the template 615, additional primitives 
can be used for rasterizing and they do not necessarily have to encompass 

10 the entire template 615. Further, any suitable transform can be used. For 
example, powerful transforms, such as perspective transforms, cab be 
applied to the template, as well as the affine transform depicted in FIG. 6C. 

The foregoing description of the invention has been presented for the 
purposes of illustration and description. It is not intended to be exhaustive 

15 or to limit the invention to the precise form disclosed. Many modifications 
and variations are possible in light of the above teaching. For example, 
any pre-processing transformation, such as filtering, equalization, or edge 
or feature finding, could be applied to the images before they are input to 
the system and method of the present invention. Thus, it is intended that 

20 the scope of the invention be limited not by this detailed description, but 
rather by the claims appended hereto. 
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WHAT IS CLAIMED IS: 

1 . A method for comparing and matching a first set of digital data 
to at least a second set of digital data, comprising: 

raster transforming at least one of the first set of digital data 
5 and the second set of digital data; and 

statistically comparing and matching the raster transformed 
sets of digital data to appropriately corresponding portions of each other. 

2. The method of claim 1, further comprising analyzing the 
10 statistical comparisons and generating new transformations for matching 

the sets of data. 



3. The method of claim 1, further comprising statistically 
comparing the raster transformed sets of digital data until a match or non- 
15 match between the first and second sets of data is achieved. 



4. The method of claim 1, wherein the raster transforming 
comprises raster transforming at least one of the first or the second set of 
digital data and computing statistics on the transformation. 

20 

5. The method of claim 4, wherein statistically comparing and 
matching comprises analyzing the computed statistics of the transformation 
and calculating new and different transformations on the digital data. 

25 6. A method for comparing and matching a first set of digital data 

to at least a second set of digital data, comprising: 

loading at least one of the first and second sets of digital data 
into a first memory device; 
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rendering model transformations and accumulating statistics of 

the loaded digital data; 

adjusting the models based on the accumulated statistics; and 
statistically comparing and matching the model 
5 transformations of the loaded set of digital data to appropriately 

corresponding portions of the other set of digital data. 

7. The method of claim 6, further comprising statistically 
comparing the sets of digital data until a match or non-match between the 

10 first and second sets of data is achieved. 

8. The method of claim 6, wherein adjusting the models 
comprises analyzing the statistical comparisons and generating new 
transformations for matching the sets of data. 

15 

9. A system for tracking digital templates of a digital scene 
defined by plural images, comprising: 

a raster processor that transforms at least one of the 
templates; and 

20 a compare processor that simultaneously and statistically 

compares and matches images associated with the templates for tracking 
the templates. 

10. The system for tracking digital templates of claim 9, wherein 
25 the compare processor comprises an address generator that generates 

addresses for the template and the image that are to be compared. 

1 1 . The system for tracking digital templates of claim 9, wherein 

20 



the addresses reflect transformations, including combinations of rotations, 
scales and perspective transforms of the template or image. 

12. The system for tracking digital templates of claim 9, wherein 
5 the addresses serve as input to filtering functions that read from the images 

to be compared and generate color values. 

13. The system for tracking digital templates of claim 9, wherein 
the template comprises a group of pixels of the image and wherein the 

10 compare processor comprises an acceptance tester preprogrammed to 
decide whether to allow a pixel of the template to contribute to the 
statistics. 

14. The system for tracking digital templates of claim 13, wherein 
15 If the pixel is permitted to contribute, the color values are sent to a 

statistics/comparison device for statistical analyses and comparison 
processing. 

15. The system for tracking digital templates of claim 14, wherein 
20 the statistics/comparison device contains variables that are updated for 

each pixel based on the input color values. 

16. The system for tracking digital templates of claim 15, wherein 
the statistical analyses compares and matches the template to the image 

25 by initially defining a function that estimates the similarity between the 
template and the image. 

17. The system for tracking digital templates of claim 16, wherein 
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the template is located in the image by computing the function at various 
locations in the image and determining where the function is maximized. 

18. The system for tracking digital templates of claim 9, wherein 
5 the compare processor comprises an alpha blending device that allows use 
of an additional color component that corresponds to the opacity of a 
surface for controlling the amount of color of a pixel in the source surface to 
be blended with a pixel in the destination surface. 

10 1 9. The system for tracking digital templates of claim 9, wherein 

the raster processor renders the template at a plurality of offsets for 
allowing the raster processor to at least one of determining a desired 
position for the template and accumulate information to analytically 
compute a desired update. 

15 

20. The system for tracking digital templates of claim 1 9, wherein 
the offsets are fractional perturbations to vertices of the templates. 
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A SYSTEM AND METHOD FOR STATISTICALLY COMPARING 
5 AND MATCHING PLURAL SETS OF DIGITAL DATA 



ARRTRACT OF THF INVFNTinN 
The present invention is embodied in a system and method for 
10 statistically comparing a first set of digital data to at least a second set of 
digital data and matching the first set of digital data to appropriately 
corresponding portions of the second set of digital data. The first or the 
second set of digital data can be transformed during statistical analysis to 
enhance statistical analysis of the digital data. 
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RECEIVE A FIRST SET OF DIGITAL DATA 
AS A FIRST INPUT AND/OR RECEIVE A 
SECOND SET OF DIGITAL DATA AS A 
SECOND INPUT 
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RASTER TRANSFORM EITHER THE FIRST 
OR THE SECOND SET OF DIGITAL DATA 
AND COMPUTE STATISTICS ON THE 
TRANSFORMATION 



ANALYZE THE RESULTING STATISTICS 
AND MAKE CALCULATED 
DETERMINATIONS BASED ON THE 

RESULTING STATISTICS FOR 
GENERATING NEW AND DIFFERENT 
TRANSFORMATIONS ON THE THE FIRST 
OR THE SECOND SET OF DIGITAL DATA 
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PERFORM UNTIL A MATCH OR NON- 
MATCH BETWEEN THE FIRST AND THE 
SECOND SET OF DIGITAL DATA IS 
ACHIEVED OR BETWEEN OTHER 
DEFINED SETS OF DATA 
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SECOND MEMORY DEVICE 
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As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name. I believe I am 
the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor 
(if plural names are listed below) of the subject matter which is claimed and for which a patent is sought 
on the invention entitled: 

A SYSTEM AND METHOD FOR STATISTICALLY COMPARING AND MATCHING 

PLURAL SETS OF DIGITAL DATA 

Jffhis declaration is of the following type: 

i: [X] original 

|i [ ] divisional 

111 [ ] continuation 

111 [ ] continuation-in-part 

i 

Hie specification of which 

[lj:heck one) XX is attached hereto 

%^ was filed on 

IJ Application Serial No. 

|S| and was amended on 

(if applicable) 

I hereby state that I have reviewed and understand the contents of the above-identified specification, 
including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to the examination of this application in 
accordance with Title 37, Code of Federal Regulations §1 .56(a). 

I do not know and do not believe that the invention was ever known or used in 
the United States of America before my or our invention thereof; 

I do not know and do not believe that the invention was ever patented or 

described in any printed publication in any country before my or our invention thereof or more 
than one year prior to this application; 

I do not know and do not believe that the invention was in public use or on 

sale in the United States of America more than one year prior to this application. 
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The invention has not been patented or made the subject of any inventor's 

certificate issued before the date of this application in any country foreign to the United States of 
America on an application filed by me or my legal representatives or assigns more than twelve 
months prior to this application; and 

As to applications for patents or inventor's certificate on the invention 

filed in any country foreign to the United States of America, prior to this application by me or my 
legal representatives or assign: 
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.any foreign application(s) for patent or inventor's certificate or any PCT international application(s) 
designating at least one country other than the United States of America filed by me on the same 
Jlubject matter having a filing date before that of the application(s) of which priority is claimed. 

Ja. Prior foreign/PCT application(s) filed v\^ithin 12 mos. (6 mos. for design) prior to this 
aapplication, and any priority claims under 35 U.S-C. §119 

^ Country/PCT Application No. Date Filed Priority Claimed 
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B. All foreign application(s), if any, filed more than 12 mos. (6 mos. for design) prior to this 
U.S. application 
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Application No.: 
Filing Date: 



PRIOR U.S. APPLICATION(S) FOR WHICH BENEFIT 
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